Multi-modality deep learning model reaches high prediction accuracy in the diagnosis of ovarian cancer.

We evaluated the diagnostic performance of a multimodal deep-learning (DL) model for ovarian mass differential diagnosis. This single-center retrospective study included 1,054 ultrasound (US)-detected ovarian tumors (699 benign and 355 malignant). Patients were randomly divided into training (n = 675), validation (n = 169), and testing (n = 210) sets. The model was developed using ResNet-50. Three DL-based models were proposed for benign-malignant classification of these lesions: single-modality model that only utilized US images; dual-modality model that used US images and menopausal status as inputs; and multi-modality model that integrated US images, menopausal status, and serum indicators. After 5-fold cross-validation, 210 lesions were tested. We evaluated the three models using the area under the curve (AUC), accuracy, sensitivity, and specificity. The multimodal model outperformed the single- and dual-modality models with 93.80% accuracy and 0.983 AUC. The Multimodal ResNet-50 DL model outperformed the single- and dual-modality models in identifying benign and malignant ovarian tumors.

Diagnostic Performance of Deep Learning in Video-Based Ultrasonography for Breast Cancer: A Retrospective Multicentre Study.

OBJECTIVE: Although ultrasound is a common tool for breast cancer screening, its accuracy is often operator-dependent. In this study, we proposed a new automated deep-learning framework that extracts video-based ultrasound data for breast cancer screening. METHODS: Our framework incorporates DenseNet121, MobileNet, and Xception as backbones for both video- and image-based models. We used data from 3907 patients to train and evaluate the models, which were tested using video- and image-based methods, as well as reader studies with human experts. RESULTS: This study evaluated 3907 female patients aged 22 to 86 years. The results indicated that the MobileNet video model achieved an AUROC of 0.961 in prospective data testing, surpassing the DenseNet121 video model. In real-world data testing, it demonstrated an accuracy of 92.59%, outperforming both the DenseNet121 and Xception video models, and exceeding the 76.00% to 85.60% accuracy range of human experts. Additionally, the MobileNet video model exceeded the performance of image models and other video models across all evaluation metrics, including accuracy, sensitivity, specificity, F1 score, and AUC. Its exceptional performance, particularly suitable for resource-limited clinical settings, demonstrates its potential for clinical application in breast cancer screening. CONCLUSIONS: The level of expertise reached by the video models was greater than that achieved by image-based models. We have developed an artificial intelligence framework based on videos that may be able to aid breast cancer diagnosis and alleviate the shortage of experienced experts.

Sonography-based multimodal information platform for identifying the surgical pathology of ductal carcinoma in situ.

BACKGROUND: The risk of ductal carcinoma in situ (DCIS) identified by biopsy often increases during surgery. Therefore, confirming the DCIS grade preoperatively is necessary for clinical decision-making. PURPOSE: To train a three-classification deep learning (DL) model based on ultrasound (US), combining clinical data, mammography (MG), US, and core needle biopsy (CNB) pathology to predict low-grade DCIS, intermediate-to-high-grade DCIS, and upstaged DCIS. MATERIALS AND METHODS: Data of 733 patients with 754 DCIS cases confirmed by biopsy were retrospectively collected from May 2013 to June 2022 (N1), and other data (N2) were confirmed by biopsy as low-grade DCIS. The lesions were randomly divided into training (n=471), validation (n=142), and test (n = 141) sets to establish the DCIS-Net. Information on the DCIS-Net, clinical (age and sign), US (size, calcifications, type, breast imaging reporting and data system [BI-RADS]), MG (microcalcifications, BI-RADS), and CNB pathology (nuclear grade, architectural features, and immunohistochemistry) were collected. Logistic regression and random forest analyses were conducted to develop Multimodal DCIS-Net to calculate the specificity, sensitivity, accuracy, receiver operating characteristic curve, and area under the curve (AUC). RESULTS: In the test set of N1, the accuracy and AUC of the multimodal DCIS-Net were 0.752-0.766 and 0.859-0.907 in the three-classification task, respectively. The accuracy and AUC for discriminating DCIS from upstaged DCIS were 0.751-0.780 and 0.829-0.861, respectively. In the test set of N2, the accuracy and AUC of discriminating low-grade DCIS from upstaged low-grade DCIS were 0.769-0.987 and 0.818-0.939, respectively. DL was ranked from one to five in the importance of features in the multimodal-DCIS-Net. CONCLUSION: By developing the DCIS-Net and integrating it with multimodal information, diagnosing low-grade DCIS, intermediate-to high-grade DCIS, and upstaged DCIS is possible. It can also be used to distinguish DCIS from upstaged DCIS and low-grade DCIS from upstaged low-grade DCIS, which could pave the way for the DCIS clinical workflow.

A validation of an entropy-based artificial intelligence for ultrasound data in breast tumors.

BACKGROUND: The application of artificial intelligence (AI) in the ultrasound (US) diagnosis of breast cancer (BCa) is increasingly prevalent. However, the impact of US-probe frequencies on the diagnostic efficacy of AI models has not been clearly established. OBJECTIVES: To explore the impact of using US-video of variable frequencies on the diagnostic efficacy of AI in breast US screening. METHODS: This study utilized different frequency US-probes (L14: frequency range: 3.0-14.0 MHz, central frequency 9 MHz, L9: frequency range: 2.5-9.0 MHz, central frequency 6.5 MHz and L13: frequency range: 3.6-13.5 MHz, central frequency 8 MHz, L7: frequency range: 3-7 MHz, central frequency 4.0 MHz, linear arrays) to collect breast-video and applied an entropy-based deep learning approach for evaluation. We analyzed the average two-dimensional image entropy (2-DIE) of these videos and the performance of AI models in processing videos from these different frequencies to assess how probe frequency affects AI diagnostic performance. RESULTS: The study found that in testing set 1, L9 was higher than L14 in average 2-DIE; in testing set 2, L13 was higher in average 2-DIE than L7. The diagnostic efficacy of US-data, utilized in AI model analysis, varied across different frequencies (AUC: L9 > L14: 0.849 vs. 0.784; L13 > L7: 0.920 vs. 0.887). CONCLUSION: This study indicate that US-data acquired using probes with varying frequencies exhibit diverse average 2-DIE values, and datasets characterized by higher average 2-DIE demonstrate enhanced diagnostic outcomes in AI-driven BCa diagnosis. Unlike other studies, our research emphasizes the importance of US-probe frequency selection on AI model diagnostic performance, rather than focusing solely on the AI algorithms themselves. These insights offer a new perspective for early BCa screening and diagnosis and are of significant for future choices of US equipment and optimization of AI algorithms.

The research on artificial intelligence-assisted breast diagnosis often relies on static images or dynamic videos obtained from ultrasound probes with different frequencies. However, the effect of frequency-induced image variations on the diagnostic performance of artificial intelligence models remains unclear. In this study, we aimed to explore the impact of using ultrasound images with variable frequencies on AI's diagnostic efficacy in breast ultrasound screening. Our approach involved employing a video and entropy-based feature breast network to compare the diagnostic efficiency and average two-dimensional image entropy of the L14 (frequency range: 3.0-14.0 MHz, central frequency 9 MHz), L9 (frequency range: 2.5-9.0 MHz, central frequency 6.5 MHz) linear array probe and L13 (frequency range: 3.6-13.5 MHz, central frequency 8 MHz), and L7 (frequency range: 3-7 MHz, central frequency 4.0 MHz) linear array probes. The results revealed that the diagnostic efficiency of AI models differed based on the frequency of the ultrasound probe. It is noteworthy that ultrasound images acquired with different frequency probes exhibit different average two-dimensional image entropy, while higher average two-dimensional image entropy positively affect the diagnostic performance of the AI model. We concluded that a dataset with higher average two-dimensional image entropy is associated with superior diagnostic efficacy for AI-based breast diagnosis. These findings contribute to a better understanding of how ultrasound image variations impact AI-assisted breast diagnosis, potentially leading to improved breast cancer screening outcomes.

Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study.

OBJECTIVES: This study aimed to propose a deep learning (DL)-based framework for identifying the composition of thyroid nodules and assessing their malignancy risk. METHODS: We conducted a retrospective multicenter study using ultrasound images from four hospitals. Convolutional neural network (CNN) models were constructed to classify ultrasound images of thyroid nodules into solid and non-solid, as well as benign and malignant. A total of 11,201 images of 6784 nodules were used for training, validation, and testing. The area under the receiver-operating characteristic curve (AUC) was employed as the primary evaluation index. RESULTS: The models had AUCs higher than 0.91 in the benign and malignant grading of solid thyroid nodules, with the Inception-ResNet AUC being the highest at 0.94. In the test set, the best algorithm for identifying benign and malignant thyroid nodules had a sensitivity of 0.88, and a specificity of 0.86. In the human vs. DL test set, the best algorithm had a sensitivity of 0.93, and a specificity of 0.86. The Inception-ResNet model performed better than the senior physicians (p < 0.001). The sensitivity and specificity of the optimal model based on the external test set were 0.90 and 0.75, respectively. CONCLUSIONS: This research demonstrates that CNNs can assist thyroid nodule diagnosis and reduce the rate of unnecessary fine-needle aspiration (FNA). CLINICAL RELEVANCE STATEMENT: High-resolution ultrasound has led to increased detection of thyroid nodules. This results in unnecessary fine-needle aspiration and anxiety for patients whose nodules are benign. Deep learning can solve these problems to some extent. KEY POINTS: • Thyroid solid nodules have a high probability of malignancy. • Our models can improve the differentiation between benign and malignant solid thyroid nodules. • The differential performance of one model was superior to that of senior radiologists. Applying this could reduce the rate of unnecessary fine-needle aspiration of solid thyroid nodules.

The effect of image resolution on convolutional neural networks in breast ultrasound.

Purpose: The objective of this research was to investigate the efficacy of various parameter combinations of Convolutional Neural Networks (CNNs) models, namely MobileNet and DenseNet121, and different input image resolutions (REZs) ranging from 64×64 to 512×512 pixels, for diagnosing breast cancer. Materials and methods: During the period of June 2015 to November 2020, two hospitals were involved in the collection of two-dimensional ultrasound breast images for this retrospective multicenter study. The diagnostic performance of the computer models MobileNet and DenseNet 121 was compared at different resolutions. Results: The results showed that MobileNet had the best breast cancer diagnosis performance at 320×320pixel REZ and DenseNet121 had the best breast cancer diagnosis performance at 448×448pixel REZ. Conclusion: Our study reveals a significant correlation between image resolution and breast cancer diagnosis accuracy. Through the comparison of MobileNet and DenseNet121, it is highlighted that lightweight neural networks (LW-CNNs) can achieve model performance similar to or even slightly better than large neural networks models (HW-CNNs) in ultrasound images, and LW-CNNs' prediction time per image is lower.

A new xAI framework with feature explainability for tumors decision-making in Ultrasound data: comparing with Grad-CAM.

BACKGROUND AND OBJECTIVE: The value of implementing artificial intelligence (AI) on ultrasound screening for thyroid cancer has been acknowledged, with numerous early studies confirming AI might help physicians acquire more accurate diagnoses. However, the black box nature of AI's decision-making process makes it difficult for users to grasp the foundation of AI's predictions. Furthermore, explainability is not only related to AI performance, but also responsibility and risk in medical diagnosis. In this paper, we offer Explainer, an intrinsically explainable framework that can categorize images and create heatmaps highlighting the regions on which its prediction is based. METHODS: A dataset of 19341 thyroid ultrasound images with pathological results and physician-annotated TI-RADS features is used to train and test the robustness of the proposed framework. Then we conducted a benign-malignant classification study to determine whether physicians perform better with the assistance of an explainer than they do alone or with Gradient-weighted Class Activation Mapping (Grad-CAM). RESULTS: Reader studies show that the Explainer can achieve a more accurate diagnosis while explaining heatmaps, and that physicians' performances are improved when assisted by the Explainer. Case study results confirm that the Explainer is capable of locating more reasonable and feature-related regions than the Grad-CAM. CONCLUSIONS: The Explainer offers physicians a tool to understand the basis of AI predictions and evaluate their reliability, which has the potential to unbox the "black box" of medical imaging AI.

Feasibility of using AI to auto-catch responsible frames in ultrasound screening for breast cancer diagnosis.

The research of AI-assisted breast diagnosis has primarily been based on static images. It is unclear whether it represents the best diagnosis image.To explore the method of capturing complementary responsible frames from breast ultrasound screening by using artificial intelligence. We used feature entropy breast network (FEBrNet) to select responsible frames from breast ultrasound screenings and compared the diagnostic performance of AI models based on FEBrNet-recommended frames, physician-selected frames, 5-frame interval-selected frames, all frames of video, as well as that of ultrasound and mammography specialists. The AUROC of AI model based on FEBrNet-recommended frames outperformed other frame set based AI models, as well as ultrasound and mammography physicians, indicating that FEBrNet can reach level of medical specialists in frame selection.FEBrNet model can extract video responsible frames for breast nodule diagnosis, whose performance is equivalent to the doctors selected responsible frames.

How much can AI see in early pregnancy: A multi-center study of fetus head characterization in week 10-14 in ultrasound using deep learning.

PURPOSE: To investigate if artificial intelligence can identify fetus intracranial structures in pregnancy week 11-14; to provide an automated method of standard and non-standard sagittal view classification in obstetric ultrasound examination METHOD AND MATERIALS: We proposed a newly designed scheme based on deep learning (DL) - Fetus Framework to identify nine fetus intracranial structures: thalami, midbrain, palate, 4th ventricle, cisterna magna, nuchal translucency (NT), nasal tip, nasal skin, and nasal bone. Fetus Framework was trained and tested on a dataset of 1528 2D sagittal-view ultrasound images from 1519 females collected from Shenzhen People's Hospital. Results from Fetus Framework were further used for standard/non-standard (S-NS) plane classification, a key step for NT measurement and Down Syndrome assessment. S-NS classification was also tested with 156 images from the Longhua branch of Shenzhen People's Hospital. Sensitivity, specificity, and area under the curve (AUC) were evaluated for comparison among Fetus Framework, three classic DL models, and human experts with 1-, 3- and 5-year ultrasound training. Furthermore, 4 physicians with more than 5 years of experience conducted a reader study of diagnosing fetal malformation on a dataset of 316 standard images confirmed by the Fetus framework and another dataset of 316 standard images selected by physicians. Accuracy, sensitivity, specificity, precision, and F1-Score of physicians' diagnosis on both sets are compared. RESULTS: Nine intracranial structures identified by Fetus Framework in validation are all consistent with that of senior radiologists. For S-NS sagittal view identification, Fetus Framework achieved an AUC of 0.996 (95%CI: 0.987, 1.000) in internal test, at par with classic DL models. In external test, FF reaches an AUC of 0.974 (95%CI: 0.952, 0.995), while ResNet-50 arrives at AUC∼0.883, 95% CI 0.828-0.939, Xception AUC∼0.890, 95% CI 0.834-0.946, and DenseNet-121 AUC∼0.894, 95% CI 0.839-0.949. For the internal test set, the sensitivity and specificity of the proposed framework are (0.905, 1), while the first-, third-, and fifth-year clinicians are (0.619, 0.986), (0.690, 0.958), and (0.798, 0.986), respectively. For the external test set, the sensitivity and specificity of FF is (0.989, 0.797), and first-, third-, and fifth-year clinicians are (0.533, 0.875), (0.609, 0.844), and (0.663, 0.781), respectively.On the fetal malformation classification task, all physicians achieved higher accuracy and F1-Score on Fetus selected standard images with statistical significance (p < 0.01). CONCLUSION: We proposed a new deep learning-based Fetus Framework for identifying key fetus intracranial structures. The framework was tested on data from two different medical centers. The results show consistency and improvement from classic models and human experts in standard and non-standard sagittal view classification during pregnancy week 11-13+6. CLINICAL RELEVANCE/APPLICATION: With further refinement in larger population, the proposed model can improve the efficiency and accuracy of early pregnancy test using ultrasound examination.

A Comparative Study of Multiple Deep Learning Models Based on Multi-Input Resolution for Breast Ultrasound Images.

Purpose: The purpose of this study was to explore the performance of different parameter combinations of deep learning (DL) models (Xception, DenseNet121, MobileNet, ResNet50 and EfficientNetB0) and input image resolutions (REZs) (224 × 224, 320 × 320 and 488 × 488 pixels) for breast cancer diagnosis. Methods: This multicenter study retrospectively studied gray-scale ultrasound breast images enrolled from two Chinese hospitals. The data are divided into training, validation, internal testing and external testing set. Three-hundreds images were randomly selected for the physician-AI comparison. The Wilcoxon test was used to compare the diagnose error of physicians and models under P=0.05 and 0.10 significance level. The specificity, sensitivity, accuracy, area under the curve (AUC) were used as primary evaluation metrics. Results: A total of 13,684 images of 3447 female patients are finally included. In external test the 224 and 320 REZ achieve the best performance in MobileNet and EfficientNetB0 respectively (AUC: 0.893 and 0.907). Meanwhile, 448 REZ achieve the best performance in Xception, DenseNet121 and ResNet50 (AUC: 0.900, 0.883 and 0.871 respectively). In physician-AI test set, the 320 REZ for EfficientNetB0 (AUC: 0.896, P < 0.1) is better than senior physicians. Besides, the 224 REZ for MobileNet (AUC: 0.878, P < 0.1), 448 REZ for Xception (AUC: 0.895, P < 0.1) are better than junior physicians. While the 448 REZ for DenseNet121 (AUC: 0.880, P < 0.05) and ResNet50 (AUC: 0.838, P < 0.05) are only better than entry physicians. Conclusion: Based on the gray-scale ultrasound breast images, we obtained the best DL combination which was better than the physicians.

TeroKit: A Database-Driven Web Server for Terpenome Research.

Natural products are the major resource of drug discovery, and terpenoids represent the largest family of natural products. Terpenome is defined as all terpenoid-like and terpenoid-derived natural compounds, including the terpenoids, steroids, and their derivatives. Herein, aiming to navigate the chemical and biological space of terpenome, the first comprehensive database dedicated to terpenome research has been developed by collecting over 110 000 terpenome molecules from various resources, distributed in 14 351 species, belonging to 1109 families, and showing activity against 1366 biological targets. Much of the publically available information or computationally predicted properties for each terpenome molecule is annotated and integrated into TeroKit (http://terokit.qmclab.com/), serving as free Web server for academic use. Moreover, several practical toolkits, such as target profiling and conformer generation modules, are also implemented to facilitate the drug discovery of terpenome.